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Mail vs. Email - A Pulse Check on Surveys 


By: Ron Reitz and Tom Treiman 


Information Need: 

With nearly every survey Resource Science Division 
conducts, administration, budgeters, and staff ask 
“Wouldn’t online surveys save MDC money?” and we 
respond, “It depends!” Online surveys have become 
attractive as a presumably economical way to 
measure constituents’ preferences, opinions, and 
attitudes. However, issues such as non-response 
bias, sample validity, and sampling error due to 
internet access differences in the population can have 
effects on representativeness, reliability, and 
accuracy. We examined response rates, responses, 
demographics, and potential sources of bias in the 
2016 post-season firearms deer and deer hunter 
opinion surveys using 1) a mail-back paper survey 
and 2) an online survey sent to a sample of those with 
an email address in the point-of-sale (POS) system. 


Results: 

Deer permit holders with emails tended to be slightly 
younger and fewer lived in rural areas than those 
without email addresses (Table 1). Both groups were 
overwhelmingly male. 


Table 1. Permit holder information. 


2016 Deer Permit Buyers | NoEmail | Email | 
Percent Male 82.7% 83.2% 


Percent Rural 80.1% 73.8% 
Average Age (years) 


Response rates varied between 27% for the online 
firearms survey to 38% for the mail firearms survey 
(Table 2). Respondents were older, less rural, and 
more male than non-respondents. 





Table 2. Response rates by survey type and mode. 


Online surveys allow a nuanced look at response 
rates, showing that over half of recipients who opened 
the email responded (Table 3). Small percentages had 
“bad” email addresses or opted out. 


Table 3. Response rates by survey type. 


Survey Bad Opted Opened | Responded 
Email | Out Email to Email 


Opinion [21% [15% [60.7% [305% 


Looking at responses to key questions by mail vs. 
email reveals very few substantive differences. On the 
firearms survey, both mail and email groups hunted 
mostly on private land, desired a stable or increasing 
deer population, and had only small, non-substantive 
differences in perceptions of deer populations and 
whether they hunted in 2016. On the opinion survey, 
the groups showed no difference on where they 
hunted, perceptions of deer population, and only 
small, non-substantive differences in perceptions of 
MDC’s deer management success. There was a 
significant and substantive difference in support for 
the four-point regulation, with 70.1% support from the 
email survey and 55.5% from the mail survey. 





Discussion: 

When will it be appropriate to transition to different 
survey modes, like email, text, and app? These 
decisions may be driven by influencers such as the 
future of the postal service system, innovations in 
technology, decline in survey response, and public 
acceptability of unsolicited surveys. Will it be 
acceptable to systematically eliminate a portion of our 
sampling frame that may not have online access? 
Given that many survey methods are now 
experiencing low response rates and are becoming 
more costly to conduct, and 


Response Percent | Percent | Average 


7) 7) represent the population of 
a interest, it may make 


34.89, Respondent 
— Non-respondent | 81.9% | 79.5% 
Respondent 89.3% 68.6% 


For more information, contact: 
Missouri Department of Conservation 
Resource Science Division 
2901 West Truman Blvd. 
Jetterson City, MO 65109 
573-751-4115 
tesearch@mdc.mo.gov 


75.8% 
783% 
53.3% 
67.5% 





economic (and practical) 
sense to use the most 
economical method(s) 
available, as long as the 
population may be effectively 
sampled. 
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